1. Field of the Invention
The present invention is related to methodology for benchmarking computer system interacting over a network.
2. Background Art
Network performance benchmarks are invaluable tools for providing unbiased and valid information that can be used to evaluate and analyze the performance of various networks. Such performance benchmarking is helpful for a variety of purposes. For example, benchmarks allow system and application performance to be explained and/or predicted. This can be particularly useful for decision making while procuring large-scale systems.
The plethora of network protocols and technologies makes it difficult to find an appropriate benchmark for a particular situation. Examples of network technologies include technologies such as 1G, 10G, and protocols such as UDP, TCP, and UDAPL UDAPL. Benchmarks such as iPERF and netbench today are used by many developers, researchers and customers to evaluate and analyze the performance of the network technologies and protocols. Most of these benchmarks are targeted to a single objective. For example iPERF is basically used to provide performance data for TCP/IP, which makes the benchmark very specific in nature. Also the load that they generate is not always quite similar to the application's work load, thus making the data unrealistic.
Filebench, basically a benchmark framework for local file systems, is the first benchmark to introduce the concept of generic framework and application based analysis. The simple idea behind filebench, is that we can represent any application using process, threads, transactions and operations. Hence we could generate a very realistic work load. The data obtained from this are not only useful to analyze the performance but will also help us resolve the bottleneck with existing applications. For new applications that are yet to be designed, it gives a performance insight that is very helpful to design better performance efficient applications. Basically based on the concept that most of the benchmarks share similar objectives and the way of achieving them, filebench is definitely the best thought conceived and realized idea, in the field of benchmarks.
Although these prior art benchmark technologies work reasonably well, it is often the case that for each new application with novel requirements, new bench marking methodology needs to be implemented. The prior art benchmarks tend to be somewhat unrealistic, specific, and inflexible. These drawbacks not only hinder the performance analysis but also make it hard to maintain and support these benchmarks. It is often infeasible to compare the data from different benchmark technologies. Moreover, as requirements or applications change, it is typically necessary to re-implement the benchmarking tool. This often results in a significant amount of time being devoted to implementation resulting in added expense. It would be more desirable to have such resources devoted to performance analysis.
Accordingly, for at least these reasons, there is need for improved network benchmark technologies that are flexible and easy to implement.